Licence Creative Commons CC-By-NC-Sa

Animal studies indicate that learning is driven by reward and punishment, leading to an optimized behaviour in a given environment. Reinforcement learning is a formal computational model of such reward-based learning, enabling an autonomous agent to learn optimal control policies through trial and error interaction with a dynamic environment. The reinforcement learning problem can be solved by using dynamic programming methods to estimate the Q-function which represents the utility of taking action a in a given state s.
Recently, Ernst et.al introduced the Fitted Q Iteration algorithm, that makes efficient use of the data gathered from the system. While classical algorithms only use the current environment feedback to adapt the Q-function, Fitted Q Iteration implements a special form of long-term memory, such that all interaction experience can be utilized at each optimization step. This enables the agent to reflect on past decisions as new information about the system is revealed.
Fitted Q Iteration can be used with any function approximator to model the Q- function. Existing algorithms are based on regression trees, neural networks or kernel-methods. In this work the performance of different variations of the algorithm is compared on several reinforcement learning benchmark problems.

Licence Creative Commons CC-By-NC

A biologically inspired navigation system for the mobile rat-like robot nicknamed Psikharpax is presented, allowing for self-localization and autonomous navigation in an initially unknown environment. Parts of the model (e.g. the strategy selection mechanism) have been validated before in simulation, but have now been adapted to a real robot platform.
This article presents our work on the implementation of two independent navigation strategies and a strategy selection mechanism. We show how our robot can learn to choose the optimal strategy in a given situation using a Q-learning algorithm.

Licence Creative Commons CC-By-NC-ND

Robotic systems today have reached unprecedented levels of perceptual and motor abilities. One of the great challenges now is to equip these robots with cognitive systems that allow them to escape well defined industrial environments and face less constrained situations. In such contexts uncertainty can arise from at least two distinct sources – the changing and probabilistic structure of the environment itself, and periodic sensory and motor failures in the robot due to the increased challenges inherent in a less structured environment. Taking inspiration from biology, particularly from primate neurophysiology can help design cognitive systems architectures to enable robots to adapt to uncertainty and to have a satisfying performance, if not optimal, in very different situations.
The current research exploits the behavioral robustness of the primate prefrontal cortical mechanisms that have evolved to address such uncertainty. Based on known neurophysiology of the lateral prefrontal and anterior cingulate cortex (LPFC and ACC) we developed a neural network model that embodies and extend principals of reinforcement learning. The model relies on Reinforcement learning principles allowing an agent to associate reward values to particular actions and to improve its behavioral policy through trial-and-error. In addition, the model incorporates Meta-learning principles allowing the adjustment of meta- parameters of learning and action selection based on measures of the agent’s performance.
The model previously enabled us to perform model-based analysis on neurophysiological data and to identify meta-learning mechanisms in the prefrontal cortex of monkeys performing simple problem solving tasks. Here the model is used to reproduce monkey behavioral performance with a robot, and to enable the humanoid iCub to adapt to uncertainties introduced by the human during human-robot interaction. The combined results provide

Licence Creative Commons CC-By

This presentation deals with the modelling of the perception by the electric sense. The work has been realized in the context of the european project ANGELS. The goal of the project is to build an eel-like robot equipped with the electric sense and capable to split into several agents for exploration and recognition purposes. In this paper we present more specifically one model that we built for our sensors. The model is analytical and consists in replacing the real electrodes by calibrated charged spheres. We will show that the predictions are found to be in very good agreement with both the experiment and the simulator.

Licence Creative Commons CC-By

The peripheral auditory system of lizards is highly directional. The system is medially symmetrical and relatively simple in design. Due to a smaller head size with respect to the wavelength of the sounds to which it responds, the incident sound waves diffract around the lizard’s head, resulting in approximately equal sound pressure at the two ears. However, there is a phase difference between the sound waves arriving at either side, whose magnitude depends on the direction from which the sound appears to originate relative to the animal, and this small phase difference cue is converted by the system into a relatively larger difference in the perceived amplitude of the sound on either side.
In real lizards, the auditory cues from the peripheral auditory system are interpreted by the nervous system. This raises the question, how well can the peripheral auditory system represent sound direction? The perceived sound amplitudes cannot be linearly mapped to the sound direction, as they are non-
linear over the frontal region. However, a map between the two might be built via neural processes. It has been previously demonstrated with robotic models that it is possible to localize sound via such auditory cues, using a human-built decision model that maps these cues to motor control outputs [6]. In order to automate the construction of such a decision model, unsupervised learning algorithms can be used. We employ reinforcement learning in simulation, to train a Cerebellar Model Articulation Controller (CMAC) as a decision model that maps the auditory cues to sound direction.
The results of the simulations show that the CMAC is able to learn such a map in which the characteristics of the data points in the input space are correctly reflected. Furthermore, the results of both supervised and unsupervised approaches are in agreement with each other. The implementation and real-world trials of the unsupervised approach

Licence Creative Commons CC-By-NC

Water motions provide a wealth a sensory information in the aquatic world. Fish exploit this information with their lateral line. This sensory system consists of up to several thousand receptive units, the so-called neuromasts, distributed across the entire body of the animal. Neuromasts are made up of a patch of sensory hair cells similar to those in the vertebrate inner ear, and covered by a gelatinous cupula. There are two types of neuromasts, superficial neuromasts that are located on the surface of the skin, and canal neuromasts that are located within a system of subdermal canals. The canal fluid is in contact with the outside water through a series of pores. Neuromasts are directionally sensitive and respond either to water velocity (superficial neuromasts) or pressure differences (canal neuromasts). Due to its astonishing sensitivity and unique morphological design, the lateral line serves as a model system for bioinspired robotic systems. To make this system accessible to bio-engineering approaches, knowledge is required about the stimuli relevant to the lateral line, the peripheral layout of the system including location and orientation of the receptors, the bio-morphlogical properties of the receptors and the representation of hydrodynamic stimuli by the neuronal activity of nerve cells. In this presentation, I will focus mainly on the role of the lateral line for the detection, localization and neuronal representation of local hydrodynamic stimuli generated by a dipole source.

Licence Creative Commons CC-By-NC-Sa

The talk will have two main topics: 1) how energy carriers, receptive mosaic properties and imaging rules determine the information extracted by different sensory systems and 2) how species-specific repertoire of senses determines the perceptual world of an individual.

There are two main links between perceptual processes and external reality: the externally driven changes in patterns of energy stimulating different sensory mosaics and the effects of the sensory agent’s actions on such pattern.

In the first subtopic, following a line traced by the analysis of the electrosensory system of fish, I will first develop a comparative view on what and how spatial and temporal constraints are imposed by the type of energy of the sensory carrier on the images that sensory mosaics receive. This will be followed by the analysis of how receptive mosaics choose just a few dimensions of the image to represent image patterns and how dimension reduction leads to representation of qualia. Solutions for signal transduction and encoding will be revisited and grouped in few general classes observed in nature, some of them exemplified by electroreceptive agents. Third I will focus on perceptually oriented actions and classify them in three main groups according to their perceptual consequences: those changing the point of view, those providing and controlling the sensory carrier and those acting on peripheral and central sensory structures.

In the second and more general subtopic, I will develop the role of the brain as a predicting device. The importance of brain organization and the body plan on such predicting role will be stressed to achieve the conclusion that each species, and on the fine details each individual has its own way of representing reality and its own behavioral profile.

Licence Creative Commons CC-By-NC

South American electric knifefish are a leading model system within neurobiology. Recent efforts have focused on understanding their biomechanics and relating this to their neuronal sensory processing strategies. Knifefish swim by means of an undulatory fin that runs most of the length of their body, affixed to the belly. Propelling themselves with this fin enables them to keep their body relatively straight while swimming, a significant advantage for using this approach for underwater robots. In this study, we examined the basic properties of knifefish swimming through use of an undulatory robot that is similar in some key respects to the knifefish. We also explore a novel mode of locomotion discovered with the aid of the robot which allows the animal to swim vertically. Finally, we show results from an artificial active electrosensory system that has significant promise as a low-energy sensing technology for underwater vehicles.

Licence Creative Commons CC-By-NC

The conventional view is that neurons transmit information by using arate-based code in which higher values are encoded by higher firing rates. But there is a very different way of looking at computation in which precise spike timing is critical. I will describe recent work that shows that Spike-Time Dependent Plasticity (STDP) could allow neurons to become selective to patterns of spikes that repeat. Even more interestingly, it looks as though this sort of mechanism could be implemented in memristor-based electronic circuits, opening the possibility of a new generation of bioinspired robotic systems.

Licence Creative Commons CC-By-NC

Experiments conducted over the last decades have revealed the importance of optic flow cues in insect 3D navigation. We present explicit control schemes which explain how insects may take off and land, follow the terrain, respond suitably to headwind, avoid lateral obstacles and control their ground speed automatically. The concept of the optic flow regulator, a feedback control system based on optic flow sensors, is presented. A few optic flow regulators suffice to account for various insect flight patterns observed over the ground and over still water, under calm and windy conditions, and in straight or tapered corridors.These control schemes were tested in simulation and implemented onboard two types of insect-like robots, a helicopter and a hovercraft, which behaved much like insects when placed in similar environments. These robots were equipped with electro-optic OF sensors inspired by the motion sensitive neurons that we previously analyzed in the compound eye of flies, by combining single neuron recording with single photoreceptor stimulations. Whilst our robotic demonstrators serve us primarily to validate (or falsify) our working hypotheses about animal behavior, they offer interesting prospects for aerospace engineering because they require neither range sensors nor ground speed sensors.

Licence Creative Commons CC-By-NC

Flying animals have since long inspired admiration and fueled the imagination of scientists and engineers. Alongside biologists studying form and function of flapping flyers in nature the last decade has seen an impressive quantity of studies driven by engineering groups using new techniques to develop and study artificial biomimetic flapping flyers.
A recent field of investigation concerns the efficiency of flapping flyers, the major interrogation being about how natural systems optimize energy saving together with performance enhancement. In particular, the passive role of wing flexibility to increase flight efficiency through the bending of the wings while flapping has attracted a lot of attention. It is commonly agreed that this efficiency enhancement comes from the particular shape of the bent wing, which leads to a more favorable repartition of the aerodynamic forces. However, quantifying the amount of deformation is not sufficient to explain the performance enhancement that is observed; and the temporal evolution of the wing shape is also of critical importance.

Licence Creative Commons CC-By-NC

Bats exhibit extraordinary flight capabilities that arise by virtue of a variety of unique mechanical features. These flying mammals have developed powerful muscles that provide the folding and extension of their wing-membrane during flight (morphing). Although observing and gaining inspiration from these animals can provide significant insight into the physical requirements of flapping flight, it remains an engineering challenge to develop equivalently effective morphing wing vehicles. Bat wings are made of quite flexible bones that possess independently controllable joints, which make difficult to mimic their mechanistic basis of flight. By combining flapping and
morphing motions, bats can achieve an amazing level of maneuverability. Attempting to reproduce the muscles system acting on their wing’s joints requires the analysis of alternative actuation technologies more likely muscle fiber arrays instead of standard servomotor actuators. In this regard, smart materials have opened new alternatives and the potential of building simpler, lighter and smaller robotics systems.

Licence Creative Commons CC-By-NC

Researches on flapping-wing micro air vehicle (FMAV) have received great attention in recent years To this end, the low Reynolds number aerodynamics, as well as the flight dynamics and control were discussed elsewhere. This work will focus mainly on the material choice, structuring and actuation of flapping-wing robotic insect.
The main ideas of robotic insect were described: unlike the design using articulated wings, our flying system, actuated by micro-mechanism in the thorax, employs resonant flexible wings without articulations to obtain a higher energetic efficiency. Few works, up to now, suggested using MEMS technology to realize a polymer-based flying object at insect size. In this report, we will experimentally create and test novel insect prototypes made of SU-8 and/or PDMS, and employ analytical and Finite Element Method (FEM) tools to well understand insect wing motion and relevant different mechanisms such as the damping ones.

Licence Creative Commons CC-By-NC

One of the major hypotheses underlying many aerodynamic theories states that the flow near the wings can be sufficiently well described within the two-dimensional approximation. This viewpoint is classical and very appealing since it greatly simplifies the modelling. The two-dimensional approximation is well justified for propellers, but experiments with insects showed new and important three-dimensional effects. Obviously, they are also relevant to insect-like flying robots.
The three-dimensional character of the flow strongly influences the dynamics of the vortex shedding. Even though the wings operate at large angles of attack, the leading-edge vortices persist over the upper surfaces and increase the lift. This feature makes a striking contrast to the periodic vortex shedding that occurs in the two-dimensional motion. This ‘stable’ behaviour of the leading-edge vortices is accompanied by a strong flow in the spanwise direction. In the present work we explore this flow using numerical simulations and propose a very simple, but elucidating, potential flow model.

Licence Creative Commons CC-By-NC

One of the major challenges in robotics is to develop fly- sized robots that can autonomously fly around in unknown environments. The challenge derives from the fact that flying locomotion requires the robot to continuously react to its environment in real time, while the light weight of the robot significantly limits the energy, sensors, and processing onboard.
Essentially, there are two main approaches to creating fly- like robots: bottom-up and top-down. In the bottom-up approach, one starts by creating all the tiny parts that are deemed important to a fly-sized ornithopter. The most remarkable example of this approach is the 60 mg robotic insect developed at Harvard University, which can produce sufficient thrust to take off vertically. This is achieved by using Smart Composite Microstructures (SCM). The robotic insect was still fixed to taut guide wires that restricted the robot to vertical motion and provided both energy and control. In future work, the group plans to allow all degrees of freedom and to incorporate onboard energy supply, sensors, and processing.
In the top-down approach, one starts with a fully functioning (relatively large-scale) ornithopter. By studying this ornithopter, theoretical insights can be gained into the necessary properties for a smaller version. Research then progresses by creating and analyzing ever smaller systems, while always maintaining a fully functioning flying robot. One advantage of this approach is that it allows interplay between theory and practice. Especially in the field of artificial intelligence, having a physical and fully functioning robot is of great value: real-world tests force the experimenters to take into account all aspects of the robotic system. In addition, they reveal physical properties of the system that can be exploited by the algorithms.
In this extended abstract, we discuss the current state of our research on aerodynamics and autonomy

Licence Creative Commons CC-By-NC

Insect flight is now an active and well-integrated research area, attracting participation from a wide range of talents. Not only because nature has bred astonishing solutions to this complex real world challenges but also it offers us a perfect objective of biologically inspired engineering research, which could lead to a greater understanding of novel biomechanisms on the "intelligence" of a flying insect and a breakthrough for biomimetic designing and building small-scale flying mechanical devices, namely, Micro Air Vehicles (MAVs). Aiming at developing an effective tool to unveil key mechanisms in bio-flights as well as to provide guidelines for bio-inspired MAV design, we propose a comprehensive computational framework, which integrates aerodynamics, flight dynamics, vehicle stability and maneuverability. This paper highlights our recent works associated with the MAV-motivated computational biomechanics in insect flight and its application for development and evaluation of an insect-inspired, flapping-wing MAV with a weight of 2.4~3.0 g and a wingspan of 10~12 cm.

Licence Creative Commons CC-By-NC

The lamprey is one of the few vertebrates in which the neural control system for goal-directed locomotion including steering and control of body orientation is well described at a cellular and synaptic level. In this lecture I will review the extensive modelling at a large-scale level, which we are performing not only of the brainstem spinal cord networks underlying propulsion, but also the tectal mechanisms involved in steering and the forebrain mechanisms underlying selection of different aspects of motor behaviour. We are able to model the system with compartmental Hodgkin-Huxley neurons and with the approximate number of neurons that are responsible in the behaving animal (10000 neurons at spinal level). We also demonstrate how the network activity and direction of motion can be controlled by interacting with only a few out of hundred segments. This arrangement simplifies the control of motion and steering. Moreover, the data from biology and modelling is also used to control a bio-inspired lamprey robot in interaction with the Dario laboratory in Pisa.

Licence Creative Commons CC-By-NC

ADVANCES in our understanding of the neuronal locomotor circuits of salamanders and lampreys have increased our need for tools on which to evaluate hypotheses and computational models. Besides simulation, robots can be a very powerful tool in this direction. Robots can be the interface between our computational models and the real world. This is the reason why proper robotic design and advanced performance are important.
Besides Salamandra robotica, only a few other prototypes of salamander-like robots (i.e., quadruped robots with several degrees of freedom in the spine) have been the object of scientific publications:
i) Robo-Salamander, a salamander-like robot with two degrees of freedom for the spine, and two for each leg; no experiments with it have been published, and no other publications followed. It is tethered (both for power and control), and is only capable of walking.
ii) A salamander robot with 6 segments and an on-board FPGA-based control system. It is not amphibious and can only walk. None of these robots is capable of swimming, and none is fully autonomous.
We will describe our new salamander-like robot (Salamandra Robotica II) and we will demonstrate its performance through the characterization of some basic control parameters.

Licence Creative Commons CC-By

Small creatures have inherent mobility problems due to their stride limitation and small body size compared to surroundings. Meanwhile, some of them, such as locusts and fleas, have evolved to acquire better mobility by employing saltatorial (jumping) locomotion. In the similar context, the mobility of small-scale robots can be improved by jumping. To date, several small-scale jumping robots have been developed; the jumping robot Grillo is designed for long and consecutive jumping gait [1]. A steerable miniature jumping robot shows great jumping height and steering ability [2]. The closed elastica robot employs snap-through buckling for jumping [3]. They commonly employ the catapult system with passive latch; each of the mechanism lacks active trigger.
This paper presents 1g flea-inspired jumping robot. A flea is well-known for its dramatic jumping ability: 100 times as high as its body length. This outstanding performance is contributed to the special catapult system. Known as active latch and trigger, the system generates excellent catapult motion in the cramped anatomy [4][5]. This inspired us to develop extremely small and light jumping robot without conceding the performances. Shape memory alloy (SMA) spring is employed not only as actuator but also as energy storage element in the robot. In fabrication, smart composite microstructure (SCM) process is used for several advantages in small-scale robots: embedded electrical circuit design, machining efficacy, and light weight of the robot [6][7].
In the following sections, we first introduce the detailed mechanism of active latch and trigger system. Equation of motion and its numerical solution are presented to predict the theoretical jumping performance of the robot. Jumping motion of the prototype is captured by high speed camera and analyzed.

Licence Creative Commons CC-By

Animals in nature use dynamic gait to achieve fast locomotion on rough terrains. During the dynamic gait, legs make an impact on the ground when the legs touch the ground. The impact force generates a ballistic flight state: the state when all feet leave the ground. Several researches on adopting the dynamic gaits to robotic platforms have been studied; for instance, legged robots by Raibert et al., SCOUT2, and PAW. Most of the robots use pneumatic actuators to achieve the powerful impact on the ground; however, the robots have limitations of relatively large size and huge noise from the pneumatic actuators.
Objective of the research is to develop a light-weight hopping leg for the dynamic gait by using electric motors with crank-slider mechanism. Instead of using pneumatic actuators, we used an electric motor to generate the powerful impact on the ground for high hopping. A crank-slider mechanism transforms continuous rotating actuation of the electric motor to repeated linear actuation in hopping direction. A linear spring at the end of the hopping leg performs storing and releasing potential energy. The rest of the paper explains principles and experiments on the research.

Licence Creative Commons CC-By-NC

Biomimetism has been widely emerging in the last years in robots modeling and design. Many scientists turned their attention to nature for inspiration. In fact biomimicry and bio-inspiration have taken several applications in robotics. Some scientist used observations to build their robots while others have copied the morphology of biological systems in their designs. There are many successful examples of robots mimicking the mechanical design and motion trajectories of animals. However, those designs are compromised by the limitations of traditional fabrication methods and the use of stiff materials. Nature, on the other hand makes use of compliant materials frequently. Stiff structures such as bones and shells are made with non-uniform stiffness and connected by soft tissue. Recently the use of compliant materials in robotics is emerging; Examples include terrestrial locomotion (Cham, October/November, 2002) and underwater locomotion (Pablo Valdivia y Alvarado, March 2006). The present work is part of FILOSE (Robotic Fish LOcomotion and SEnsing) project. FILOSE aims to investigate how fish sense the flow around them and react to the changes in the flow patter. In this study, the vibration theories of continuous materials are used to perform desired complex body motions. The fish-like robot should imitate the kinematics of a real fish. The robot tail is modeled as a cantilever beam actuated by a time varying moment. The equations of motion relating the lateral deflection to the actuation moment are derived. A physical model for the robot is build and the corresponding design is discussed.

Licence Creative Commons CC-By

The modeling and simulation of fish-like swimming is of interest in life sciences as well as in engineering applications. Understanding the mechanics of swimming can help clarifying some aspects of the evolution and of the physiology of aquatic organisms. In engineering, the study and optimisation of aquatic locomotion can improve the design of underwater vehicles having superior manouvering capabilities. The aim of this study is to present a simulation technique that is an extension to moving objects of an existing method [1] highlighted onto fish like swimming in 2D and 3D. In particular we are concerned with the power required for swimming, for a single fish and for small fish school.

Licence Creative Commons CC-By-NC

Researches on animal locomotion in fluid have now a long history. Focusing on the area of Mathematical Physics, the modeling leads to a system of PDEs (governing the fluid flow) coupled with a system of ODEs (driving the rigid motion of the immersed body). The first difficulty mathematicians came up against was to prove the well-posedness of such systems. This task was carried out in different papers.
Once the well-posedness of the fluid-swimmer dynamics has been established, the following step was to investigate its controllability. About this topic, still very few theoretical results are available: the authors prove that a 3D three-sphere mechanism, swimming along a straight line in a viscous fluid is controllable. We prove that a 2D shape changing body swimming in a potential flow can track approximately any given trajectory.
Some authors are rather interested in describing the dynamics of swimming in terms of Geometric Mechanics. We refer to articles for references in this area.
In this note, we consider a 3D shape changing body swimming in a potential flow. Under some symmetry assumptions (the swimmer is alone in the fluid and the fluid-swimmer system fills the whole space) we shall prove a generic controllability result, generalizing and improving what has been obtained for a particular 2D model.

Licence Creative Commons CC-By-NC-ND

Animals are fascinating for their abilities to displace in severe conditions with high performances in terms of agility and efficiency. This explains why roboticists today investigate a new generation of robots bio-insipired from snakes, moths, fishes, etc. For the bio-roboticist studying locomotion, the question is: what are the mechanisms that animals have discovered along evolution in order to solve the hard difficulties that robot designers and controllers face? For this purpose, generic tools of dynamic modelling and analysis are useful for roboticists and bio-mecanicists. The presentation has two objectives. First, it addresses the qualitative question of the classification of locomotor models and second, it gives efficient algorithmic tools for their computations. From the second objective, we will propose a general algorithm which can capture the dynamics of any multibody system realized by the connection of wheeled/unwheeled bodies. Enter into this class of systems, the over-constrained systems as are the snake like robots, the unconstrained systems as the walking robots in their flying phase or any underactuated nonholonomic system as the trike or the snake board...